from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.10.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.098	0.033	0.0	0.019	1	5	0.714	2.057	0.054	9.285	0.243	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.098	0.033	0.0	0.019	1	5	0.714	0.019	0.000	983.802	22.454	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.330	0.013	0.0	0.330	1	5	1.000	2.057	0.054	0.160	0.008	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.330	0.013	0.0	0.330	1	5	1.000	0.019	0.000	16.976	0.767	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.143	0.191	0.0	0.019	-1	1	0.646	1.891	0.039	10.122	0.234	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.143	0.191	0.0	0.019	-1	1	0.646	0.023	0.002	827.468	69.190	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.330	0.013	0.0	0.330	-1	1	0.000	1.891	0.039	0.175	0.008	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.330	0.013	0.0	0.330	-1	1	0.000	0.023	0.002	14.271	1.313	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.170	0.134	0.0	0.019	1	1	0.646	1.208	0.017	15.865	0.248	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.170	0.134	0.0	0.019	1	1	0.646	0.019	0.000	1003.039	23.147	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.352	0.011	0.0	0.352	1	1	0.000	1.208	0.017	0.291	0.010	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.352	0.011	0.0	0.352	1	1	0.000	0.019	0.000	18.421	0.718	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.518	0.307	0.0	0.020	1	100	0.836	2.066	0.022	9.448	0.179	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.518	0.307	0.0	0.020	1	100	0.836	0.023	0.007	846.320	271.418	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.331	0.009	0.0	0.331	1	100	0.000	2.066	0.022	0.160	0.005	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.331	0.009	0.0	0.331	1	100	0.000	0.023	0.007	14.340	4.610	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.658	0.153	0.0	0.020	-1	5	0.714	2.732	0.027	7.195	0.091	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.658	0.153	0.0	0.020	-1	5	0.714	0.024	0.003	808.368	96.277	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.334	0.012	0.0	0.334	-1	5	1.000	2.732	0.027	0.122	0.004	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.334	0.012	0.0	0.334	-1	5	1.000	0.024	0.003	13.743	1.704	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.577	0.332	0.0	0.020	-1	100	0.836	2.786	0.054	7.027	0.181	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	19.577	0.332	0.0	0.020	-1	100	0.836	0.024	0.002	809.260	58.019	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.330	0.009	0.0	0.330	-1	100	0.000	2.786	0.054	0.118	0.004	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.330	0.009	0.0	0.330	-1	100	0.000	0.024	0.002	13.638	1.017	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.057	0.036	0.0	0.004	1	5	0.882	2.139	0.031	1.897	0.032	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.057	0.036	0.0	0.004	1	5	0.882	0.003	0.000	1542.300	99.384	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.283	0.007	0.0	0.283	1	5	1.000	2.139	0.031	0.132	0.004	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.283	0.007	0.0	0.283	1	5	1.000	0.003	0.000	107.416	7.289	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.121	0.089	0.0	0.004	-1	1	0.844	1.746	0.013	2.360	0.054	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.121	0.089	0.0	0.004	-1	1	0.844	0.009	0.002	470.345	132.906	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.276	0.007	0.0	0.276	-1	1	1.000	1.746	0.013	0.158	0.004	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.276	0.007	0.0	0.276	-1	1	1.000	0.009	0.002	31.511	8.909	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.054	0.063	0.0	0.004	1	1	0.844	1.063	0.013	3.813	0.075	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.054	0.063	0.0	0.004	1	1	0.844	0.002	0.000	2066.146	233.059	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.270	0.007	0.0	0.270	1	1	1.000	1.063	0.013	0.254	0.007	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.270	0.007	0.0	0.270	1	1	1.000	0.002	0.000	137.432	15.766	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.206	0.050	0.0	0.004	1	100	0.898	2.159	0.023	1.948	0.031	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.206	0.050	0.0	0.004	1	100	0.898	0.003	0.000	1577.764	114.251	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.274	0.006	0.0	0.274	1	100	1.000	2.159	0.023	0.127	0.003	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.274	0.006	0.0	0.274	1	100	1.000	0.003	0.000	102.697	7.680	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.108	0.081	0.0	0.004	-1	5	0.882	2.840	0.034	1.447	0.033	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.108	0.081	0.0	0.004	-1	5	0.882	0.009	0.003	481.018	161.734	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.276	0.011	0.0	0.276	-1	5	1.000	2.840	0.034	0.097	0.004	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.276	0.011	0.0	0.276	-1	5	1.000	0.009	0.003	32.299	10.911	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.190	0.082	0.0	0.004	-1	100	0.898	2.785	0.032	1.505	0.034	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	4.190	0.082	0.0	0.004	-1	100	0.898	0.010	0.004	426.116	188.668	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.285	0.012	0.0	0.285	-1	100	1.000	2.785	0.032	0.102	0.005	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.285	0.012	0.0	0.285	-1	100	1.000	0.010	0.004	28.974	12.875	See	See

KNeighborsClassifier_kd_tree

onnx (1.10.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	130.324	0.000	0.0	0.130	1	100	0.971	5.326	0.082	24.471	0.375	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	130.324	0.000	0.0	0.130	1	100	0.971	0.004	0.001	31433.341	4304.789	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.942	0.065	0.0	2.942	1	100	1.000	5.326	0.082	0.552	0.015	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.942	0.065	0.0	2.942	1	100	1.000	0.004	0.001	709.649	98.454	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	125.155	0.000	0.0	0.125	-1	5	0.971	0.963	0.022	130.011	2.927	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	125.155	0.000	0.0	0.125	-1	5	0.971	0.003	0.001	36454.502	6923.836	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.004	0.066	0.0	3.004	-1	5	1.000	0.963	0.022	3.120	0.099	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	3.004	0.066	0.0	3.004	-1	5	1.000	0.003	0.001	874.894	167.294	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	125.663	0.000	0.0	0.126	-1	1	0.964	0.511	0.012	245.843	5.579	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	125.663	0.000	0.0	0.126	-1	1	0.964	0.003	0.000	42875.096	6812.936	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.960	0.091	0.0	2.960	-1	1	1.000	0.511	0.012	5.790	0.221	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.960	0.091	0.0	2.960	-1	1	1.000	0.003	0.000	1009.805	163.422	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	126.346	0.000	0.0	0.126	1	5	0.971	1.555	0.024	81.242	1.279	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	126.346	0.000	0.0	0.126	1	5	0.971	0.001	0.000	93703.034	23925.743	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.959	0.104	0.0	2.959	1	5	1.000	1.555	0.024	1.903	0.073	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.959	0.104	0.0	2.959	1	5	1.000	0.001	0.000	2194.811	565.672	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	130.617	0.000	0.0	0.131	1	1	0.964	0.827	0.013	158.006	2.468	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	130.617	0.000	0.0	0.131	1	1	0.964	0.001	0.000	103976.803	34302.331	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.994	0.097	0.0	2.994	1	1	1.000	0.827	0.013	3.622	0.130	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.994	0.097	0.0	2.994	1	1	1.000	0.001	0.000	2383.534	790.127	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	129.424	0.000	0.0	0.129	-1	100	0.971	3.186	0.040	40.618	0.506	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	129.424	0.000	0.0	0.129	-1	100	0.971	0.006	0.001	22325.753	3595.027	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.970	0.072	0.0	2.970	-1	100	1.000	3.186	0.040	0.932	0.025	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.970	0.072	0.0	2.970	-1	100	1.000	0.006	0.001	512.413	83.445	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.072	0.004	0.0	0.000	1	100	0.889	0.039	0.001	1.839	0.110	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.072	0.004	0.0	0.000	1	100	0.889	0.001	0.000	89.836	19.833	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	100	1.000	0.039	0.001	0.154	0.007	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	100	1.000	0.001	0.000	7.528	1.635	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.043	0.001	0.0	0.000	-1	5	0.897	0.026	0.001	1.660	0.077	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.043	0.001	0.0	0.000	-1	5	0.897	0.004	0.003	9.685	5.824	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	5	1.000	0.026	0.001	0.234	0.013	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	5	1.000	0.004	0.003	1.366	0.823	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.043	0.001	0.0	0.000	-1	1	0.886	0.024	0.001	1.789	0.090	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.043	0.001	0.0	0.000	-1	1	0.886	0.003	0.000	16.184	2.159	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	1	1.000	0.024	0.001	0.252	0.016	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	1	1.000	0.003	0.000	2.284	0.316	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.045	0.001	0.0	0.000	1	5	0.897	0.023	0.000	1.970	0.054	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.045	0.001	0.0	0.000	1	5	0.897	0.001	0.000	62.242	11.901	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	5	1.000	0.023	0.000	0.257	0.008	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	5	1.000	0.001	0.000	8.113	1.556	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.042	0.001	0.0	0.000	1	1	0.886	0.022	0.001	1.965	0.090	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.042	0.001	0.0	0.000	1	1	0.886	0.001	0.000	55.338	15.530	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	1	1.000	0.022	0.001	0.285	0.019	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	1	1.000	0.001	0.000	8.032	2.285	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.072	0.002	0.0	0.000	-1	100	0.889	0.041	0.003	1.743	0.129	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.072	0.002	0.0	0.000	-1	100	0.889	0.003	0.001	26.424	5.362	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	100	1.000	0.041	0.003	0.148	0.012	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	100	1.000	0.003	0.001	2.247	0.462	See	See

HistGradientBoostingClassifier_best

onnx (1.10.0) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	n_iter	mean_duration	std_duration	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.520	0.022	0.002	0.001	0.823	0.131	0.002	3.959	0.176	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.520	0.022	0.002	0.001	0.823	0.019	0.001	26.956	1.431	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.423	0.015	0.000	0.423	0.000	0.131	0.002	3.217	0.121	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.423	0.015	0.000	0.423	0.000	0.019	0.001	21.908	1.039	See	See